Distributed Newton Method for Regularized Logistic Regression
نویسندگان
چکیده
Regularized logistic regression is a very successful classification method, but for large-scale data, its distributed training has not been investigated much. In this work, we propose a distributed Newton method for training logistic regression. Many interesting techniques are discussed for reducing the communication cost. Experiments show that the proposed method is faster than state of the art approaches such as alternating direction method of multipliers (ADMM).
منابع مشابه
Distributed Newton Methods for Regularized Logistic Regression
Regularized logistic regression is a very useful classification method, but for large-scale data, its distributed training has not been investigated much. In this work, we propose a distributed Newton method for training logistic regression. Many interesting techniques are discussed for reducing the communication cost and speeding up the computation. Experiments show that the proposed method is...
متن کاملAn inexact subsampled proximal Newton-type method for large-scale machine learning
We propose a fast proximal Newton-type algorithm for minimizing regularized finite sums that returns an -suboptimal point in Õ(d(n + √ κd) log( 1 )) FLOPS, where n is number of samples, d is feature dimension, and κ is the condition number. As long as n > d, the proposed method is more efficient than state-of-the-art accelerated stochastic first-order methods for non-smooth regularizers which r...
متن کاملTraining L1-Regularized Models with Orthant-Wise Passive Descent Algorithms
The `1-regularized sparse model has been popular in machine learning society. The orthant-wise quasi-Newton (OWL-QN) method is a representative fast algorithm for training the model. However, the proof of the convergence has been pointed out to be incorrect by multiple sources, and up until now, its convergence has not been proved at all. In this paper, we propose a stochastic OWL-QN method for...
متن کاملBundle CDN: A Highly Parallelized Approach for Large-Scale ℓ1-Regularized Logistic Regression
Parallel coordinate descent algorithms emerge with the growing demand of large-scale optimization. In general, previous algorithms are usually limited by their divergence under high degree of parallelism (DOP), or need data pre-process to avoid divergence. To better exploit parallelism, we propose a coordinate descent based parallel algorithm without needing of data pre-process, termed as Bundl...
متن کاملDistributed Optimization for Non-Strongly Convex Regularizers
We develop primal-dual algorithms for distributed training of linear models in the Spark framework. We present the ProxCoCoA+ method which represents a generalization of the CoCoA+ algorithm and extends it to the case of general strongly convex regularizers. A primal-dual convergence rate analysis is provided along with an experimental evaluation of the algorithm on the problem of elastic net r...
متن کامل